Frame-Wise Cross-Modal Matching for Video Moment Retrieval

Video moment retrieval targets at retrieving a golden in video for given natural language query. The main challenges of this task include 1) the requirement accurately localizing (i.e., start time and end of) relevant an untrimmed stream, 2) bridging semantic gap between textual query contents. To tackle those problems, early approaches adopt sliding window or uniform sampling to collect clips first then match each clip with identify clips. Obviously, these strategies are time-consuming often lead unsatisfied accuracy localization due unpredictable length moment. avoid limitations, researchers recently attempt directly predict boundaries without generate first. One mainstream approach is multimodal feature vector target frames (e.g., concatenation) use regression upon boundary detection. Although some progress has been achieved by approach, we argue that methods have not well captured cross-modal interactions frames. In paper, propose Attentive Cross-modal Relevance Matching (ACRM) model which predicts temporal based on interaction modeling two modalities. addition, attention module introduced automatically assign higher weights words richer cues, considered be more important finding Another contribution additional predictor utilize internal training improve accuracy. Extensive experiments public datasets TACoS Charades-STA demonstrate superiority our method over several state-of-the-art methods. Ablation studies also conducted examine effectiveness different modules ACRM model.